Cos 511: Theoretical Machine Learning 1 Review of Last Lecture 2 Randomized Weighted Majority Algorithm (rwma)
نویسنده
چکیده
Recall the online learning model we discussed in the previous lecture: N = # experts For t = 1, 2, . . . , T rounds: 1) each expert i, 1 ≤ i ≤ N , makes a prediction ξi ∈ {0, 1} 2) learner makes a prediction ŷ ∈ {0, 1} 3) observe outcome y ∈ {0, 1} (a mistake happens if ŷ 6= y) With this framework in hand, we investigated a particular algorithm, Weighted Majority Algorithm (WMA), as follows: N = # experts Initially wi = 1, 1 ≤ i ≤ N For t = 1, 2, . . . , T rounds: 1) each expert i, 1 ≤ i ≤ N , makes a prediction ξi ∈ {0, 1} 2) calculate q0 = ∑
منابع مشابه
COS 511 : Theoretical Machine Learning
In the last few lectures, we talked about various kinds of online learning problems. We started with the simplest case in which there is a perfect expert, who is always right. In the last lecture, we generalize this into the case in which we might not have a “perfect expert”, but instead we have a panel or subcommittee of experts, whose combined prediction is always right. We also introduced th...
متن کاملCOS 511 : Theoretical Machine Learning
Last class, we discussed an analogue for Occam’s Razor for infinite hypothesis spaces that, in conjunction with VC-dimension, reduced the problem of finding a good PAClearning algorithm to the problem of computing the VC-dimension of a given hypothesis space. Recall that VC-dimesion is defined using the notion of a shattered set, i.e. a subset S of the domain such that ΠH(S) = 2 |S|. In this le...
متن کاملTheoretical Machine Learning Cos 511 Lecture #9
In this lecture we consider a fundamental property of learning theory: it is amenable to boosting. Roughly speaking, boosting refers to the process of taking a set of rough “rules of thumb” and combining them into a more accurate predictor. Consider for example the problem of Optical Character Recognition (OCR) in its simplest form: given a set of bitmap images depicting hand-written postal-cod...
متن کاملOnline Learning from Experts: Minimax Regret
In the last three lectures we have been discussing the online learning algorithms where we receive the instance x and then its label y for t = 1, ..., T . Specifically in the last lecture we talked about online learning from experts and online prediction. We saw many algorithms like Halving algorithm, Weighted Majority (WM) algorithm and lastly Weighted Majority Continuous (WMC) algorithm. We a...
متن کامل